Modern automotive infotainment systems necessitate intelligent and adaptive solutions to manage frequent User Interface (UI) updates and diverse design variations. This work introduces a vision-language framework to facilitate the understanding of and interaction with automotive UIs, enabling seamless adaptation across different UI designs. To support research in this field, AutomotiveUI-Bench-4K, an open-source dataset comprising 998 images with 4,208 annotations, is also released. Additionally, a data pipeline for generating training data is presented. A Molmo-7B-based model is fine-tuned using Low-Rank Adaptation (LoRa), incorporating generated reasoning along with visual grounding and evaluation capabilities. The fine-tuned Evaluative Large Action Model (ELAM) achieves strong performance on AutomotiveUI-Bench-4K (model and dataset are available on Hugging Face). The approach demonstrates strong cross-domain generalization, including a +5.6% improvement on ScreenSpot over the baseline model. An average accuracy of 80.8% is achieved on ScreenSpot, closely matching or surpassing specialized models for desktop, mobile, and web, despite being trained primarily on the automotive domain. This research investigates how data collection and subsequent fine-tuning can lead to AI-driven advancements in automotive UI understanding and interaction. The applied method is cost-efficient, and fine-tuned models can be deployed on consumer-grade GPUs.

Nothing beats the experience of powering down the highway in your car with the speakers blaring out your favourite tunes. But often the music doesn't match up to the moments of the drive – whether it's the chorus kicking in when you hit the accelerator or steady beats breaking up the monotony of the motorway. Now, a solution has come from an unlikely source – will.i.am, the entrepreneur and musician best known as the founder of the Black Eyed Peas. He's partnered with German car maker Mercedes Benz on futuristic in-car software called Sound Drive that'turns your car into a musical instrument'. When the driver accelerates, brakes or turns, the software reacts to create new sounds or remix existing tunes, making the driver'the conductor' and the car'the orchestra'.

Vehicle-based voice assistants are the next great frontier, incorporating artificial intelligence into the driving experience. At CES 2024, Amazon and BMW announced a partnership to significantly improve the pre-existing experience, marrying a large language model (LLM) with Alexa and the actual driver's manual. You can ask the Alexa-powered chatbot anything about your car and receive accurate real-time information. That thick and unwieldy manual can stay in the glovebox, for good. Amazon says this tool offers a "more natural way of getting to know your new car."

AI is literally everywhere, so it's not a big surprise to learn that Volkswagen is planning to bring ChatGPT to its vehicles. As part of its CES 2024 announcements, the automaker says that its existing IDA voice assistant will work with ChatGPT across a range of its newer models. VW isn't the first to try this -- Mercedes-Benz announced ChatGPT integration in June of last year, so it seems like this is certainly a thing we're all going to have to get used to. Specifically, VW says that ChatGPT will be enabled in these specific models with the latest generation of the company's infotainment systems: ID.7 (pictured above), ID.4, It'll roll out ChatGPT as as "standard feature" in "many" production vehicles in Q2 of 2024; the company didn't say in which regions, but notes that the feature is only currently "being considered" for the US market.

Researchers say they have found a hardware exploit with Tesla's infotainment system that could unlock paid upgrades for free, including Full Self-Driving (FSD) and heated rear seats. They used a technique called voltage glitching, which involves tinkering with the supply voltage of the infotainment system's processor. "If we do it at the right moment, we can trick the CPU into doing something else," Christian Werling told TechCrunch. "It has a hiccup, skips an instruction and accepts our manipulated code. That's basically what we do in a nutshell."

It may look a little like the inside of the Back to the Future DeLorean or KITT from Knight Rider, but this is actually a real concept by car brand Chrysler. The firm Stellantis unveiled the two-seater cockpit design at the Consumer Electronics Show 2023 in Las Vegas. It features a 37-inch glass'infotainment system', an AI assistant that can plan routes based on your calendar and'wellness experiences' that include meditation, karaoke and a DJ game. The AI assistant, known as MyDay, would provide a'welcome' to the driver based on biometric recognition, while also updating them on the vehicle's charge status and the weather forecast. There is still no sign of what Chrysler's first electric vehicle will look like on the outside, but this concept offers of glimpse of how their cars of the future may appear on the inside.

BMW began incorporating smart voice features into its infotainment systems using Amazon's Alexa in 2018. In the intervening years, the number of models sporting the digital assistant have only increased. At Amazon's 2022 Devices & Services Event on Wednesday, the two companies announced a deepening of their partnership: BMW's next-generation of infotainment systems will feature an Alexa-based assistant specifically developed with the driver in mind. The as-of-yet unnamed BMW assistant will be constructed from an Alexa Custom Assistant, "a comprehensive solution that makes it easy for BMW and other brands and device makers to create their own custom intelligent assistant tailored to their brand personality and customer needs." Those capabilities might include a proactive notification from the vehicle's assistant alerting the driver that the battery charge is low while automatically reserving a charging slot at the next off-ramp or preemptively scheduling regular service with the local dealership, and "will enable an even more natural dialogue between driver and vehicle," per a Wednesday BMW press release.

This, combined with major developments in computer vision and machine learning technology, has created enormous opportunities to make life better through the collection and utilization of this video data. Potential applications here range from improved security to interactive entertainment. However, the collection and utilization of this data also entails ethical privacy concerns and the potential for unwanted intrusion into people's lives without their permission. One way to attempt to achieve the benefits of more omnipresent video collection while mitigating the intrusion on privacy is through the automatic redaction of personally identifiable information (PII). This means automatically removing or obscuring content from video data that can be used to identify an individual while maintaining as much other video data as possible. A relatively new context generating a significant amount of video data is the cabins of automobiles.

This article was published as a part of the Data Science Blogathon. Many a time while driving, we need to adjust things like in-vehicle seat relaxation function, interior lighting, music, air-conditioning, fragrances, etc, manually while attempting to keep our eyes on the road. However, it is very inconvenient and sometimes it can potentially put the driver and the co-passengers at the risk. Furthermore, it becomes even more complex when a previous driver has different preferences for cabin temperature, audio playlist, and so on. In light of this, would not it be convenient if our vehicle's heating, ventilation, illumination, music, visor, and air conditioning (HVAC) system could learn our individual preferences, and automatically make these adjustments for us?

Google unveiled Wednesday at CES 2022 a range of new ways to keep its Android devices connected -- and that includes cars. As more vehicles go electric and automakers evolve into software developers, expect to see more plays directed at turning cars into connected devices. Take Volvo Cars, for instance. The automaker and Google announced at CES 2022 new content and services that will be coming to future Volvo vehicles, including the ability to download and use the YouTube app via Google Play Store and the ability to communicate with the Google Home ecosystem. New Volvo car models are equipped with an Android Automotive operating system and have embedded voice-controlled Google Assistant, Google Play Store, Google Maps and other Google services into its infotainment system.